Spontaneous structure formation in a network of chaotic units 
with variable connection strengths 
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As a model of temporally evolving networks, we con- 
sider a globally coupled logistic map with variable connec- 
tion weights. The model exhibits self-organization of network 
structure, reflected by the collective behavior of units. Struc- 
tural order emerges even without any inter-unit synchroniza- 
tion of dynamics. Within this structure, units spontaneously 
separate into two groups whose distinguishing feature is that 
the first group possesses many outwardly-directed connections 
to the second group, while the second group possesses only few 
outwardly-directed connections to the first. The relevance of 
the results to structure formation in neural networks is briefiy 
discussed. 

Recently, studies of various types of networks have 
been attracting the interest of researchers from a broad 
range of scientific diciplines . Although in some such 
studies, there is a time dependence of the structure of the 
network consisting of an increase in the number of units 
of which it is composed j|] , the elements themselves are 
essentially static units, with neither their intrinsic prop- 
erties nor their interactions with each other evolving in 
time. Most networks in the real world, however, consist 
of dynamic elements, and the dynamics of the individual 
elements influence the formation of network structure. 
Thus in order to understand realistic networks, it is nec- 
essary to consider the modeling of networks with such 
dynamic elements. In this Letter we study an abstract 
model of a network composed of dynamic elements and 
report its behavior, as found through numerical simula- 
tion, focusing mainly on the formation of structure. 

We consider a network of N dynamic units that in- 
teract with each other through connections with time- 
dependent strengths. For simplicity, we describe the dy- 
namics of both the units and the connection strengths 
with discrete-time maps. Hence, our model belongs to a 
class of globally coupled maps (GCM) We denote by 
f{x) the function defining the map for the dynamics of 
each unit. With this, our model is given by the set of 
equations 



N 



(1) 



where is the state variable of the i-th unit (1 < i < 
N) at the n-th time step. The coupling c represents 
the strength of the influence of the other units on the 
dynamics of unit i (0 < c < 1), and u;^ is the time- 
dependent weight of the connection from the unit j to i 



at time step n. As the map providing the dynamics of 
the units, we adopt the logistic map f{x) = ax{l — x), 
but we believe that qualitatively similar behavior would 
be displayed by the system for any form of f(x) that 
exhibits chaos. 

With regard to the dynamics of the connection 
strengths, we stipulate that the connections between 
units i and ?' with similar values and cc:^ are 
strenghened |@,§. (This can be regarded as an extension 
of Hebb's rule, which is widely used in neural network 
studies Also, we consider there to be a resource in 
the system that is used to establish connections between 
units. Then, we assume that there is a limitation on this 
resource. As a result, there exists competetion among 
connections for this resource. Instead of using an ex- 
plicit variable representing the resource, we incorporate 
this effect into our model through the normalization of 
the connection strengths, as 



[1 + S ■ g{xl^,x3j]w'^ 
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where ^ is a parameter that represents the plasticity of 
the connection strengths, and g{x\^,xl^) is a monoton- 
ically decreasing function of the absolute value of the 
difference between its arguments, whose form we choose 



here is g(.x\,xi) = l -2\xl, 
normalization given in Eq. ( 



c:^|. Note that, due to the 
w^^ is generally not equal 



to w^^; i.e., the network is asymmetric. 

In the following, we give the results of our numerical 
simulations of the model. Throughout this Letter, the 
number of units N is set to 100, though the results de- 
scribed below do not change qualitatively for larger sys- 
tems except for the existence of a longer transient behav- 
ior. The initial conditions we used are as follows. First, 
the initial values of the self-connections were set to 0. 
Then as determined by Eq. (^), they remained at for 
n > 0. Second, all the remaining connection strengths 
were set to identical values. From the constraint of the 
normalization, this value is determined to be 1/{N — 1). 
Finally, for the state variables, the initial values were 
randomly chosen from the interval (0, 1) with a uniform 
sampling measure. 

In our model, we have three parameters: a, which con- 
trols the dynamics of each unit, c, which determines the 
overall sterngth of the interactions between the units, and 
S, which governs the connection dynamics. Here we fix 
the parameter 5 to 0.1 and study how the behavior of 
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the system changes as the function of the values of the 
parameters a and c. 

It is known that the dynamics of GCM can be classified 
into four phases, according to the degree of synchroniza- 
tion and clustering among units In contrast to the 
conventional GCM, only three of these four phases ap- 
pear in our system. The first is the coherent phase, in 
which all the units take the same value and oscillate syn- 
chronously. The second is the ordered phase, in which 
the units split into a few clusters and all the units within 
each such cluster oscillate synchronously. The third is 
the desynchronized phase, in which there is no synchro- 
nization between any two units |^ . 

Corresponding to the different types of collective be- 
havior, different network structures emerge. In the co- 
herent phase, all connections have almost identical val- 
ues and these values do not change over time. In the 
ordered phase, due to the formation of clusters of units, 
connections between units that belong to the same clus- 
ter have similar finite values, determined by the size of 
the cluster, while connections between two units from 
different clusters tend towards 0. In this case too, the 
network is static. The situation is different, however, in 
the desynchronized phase, in which connection strengths 
can change, and the network structure is not fixed over 
time. The structure of the network in this phase is com- 
plicated, but not completely random. In this Letter we 
consider only the phenomena observed in this desynchro- 
nized phase, because we are presently interested in the 
behavior of dynamic networks. This phase roughly cor- 
responds to the parameter ranges 3.7 < a < 4.0 and 
< c < 0.2. In the simulations reported in th following, 
parameter values outside these ranges were not used. 

To characterize the global behavior of the network in 
the parameter space, we define some characteristic quan- 
tities of the network and study their parameter depen- 
dence. 

First, as an index of the magnitude of the temporal 
change of the network, we define an average variation 
of the connection strength per step. We call this the 
'activity of the network', and write it 

^=(^~E E"i<-^iI' (3) 

^ ' i^j n=Tt 

where Tt is the length of the transient period and is 
the length of the measuring period. 

In FIG. 1(a), the activity A of the network is plotted 
with respect to the parameters a and c on a gray scale. 
Here Tt and t„i were chosen as 100,000 and 1,000, re- 
spectively. A broad band of high activity is seen around 
the line c = .15 x (a — 3.7), corresponding to the bright 
region in FIG. 1(a). Note that there is no synchroniza- 
tion between the dynamics of the units anywhere in the 
parameter space shown in Fig 1(a), Nevertheless, there 
is a rather wide region of quite low activity. In most 
of this region, most of th units exist in pairs, with the 
units in each such pair having non-zero connections only 



between each other, forming fixed pairs in the network. 
While the dynamics of two units forming a pair are not 
synchronized, they are highly correlated. 

In the regime of high activity, more complex and dy- 
namic network structure is formed. To observe this, 
we consider the average connection matrix, denoted as 
W^^ and defined as the temporal average of w"^^ : W^^ — 
T~ J2n=T'" '^n I where and were introduced in Eq. 

If the dynamics of the connection strength are com- 
pletely random, it is expected that the average con- 
nection strengths will take almost identical values for 
each i and j, and the variance among units will de- 
crease to as the averaging time increased. Contrast- 
ingly, if there exists structure in a network with high 
activity, there should be some variance among units in 
W"^ . Keeping this in mind, we consider the sum of the 
average connection strengths eminating from one unit: 

Wlut = E"=i^^''- We calculated the variance of W^^^ 
over i for different parameter values. The result is dis- 
played in FIG. 1(b) for r* = 100,000 and r™ = 1,000. 
We find that a large variance is observed just below the 
line c = .15 X (a - 3.7). 

Comparing FIG.s 1(a) and (b), we can see that the re- 
gion of high network activity can be decomposed into 
two regimes: one with a large variance of W^^^ [for 
c < .15 X (a — 3.7)], and the other with small W^^j [for 
c > .15 X (a — 3.7)]. As mentioned above, a large vari- 
ance in the active regime indicates the existence of some 
structural order in a temporally evolving network. 

In the following, we investigate the structural charac- 
teristics of this dynamic yet ordered network. In par- 
ticular we consider the parameter values a = 3.97 and 
c = 0.12, which correspond to the largest variance in 
the active regime However we point out that the general 
characteristics of the network do not depend sensitively 
on this special choice of the parameter values. 

First, we study the structural change over time from 
the initial all-to-all type network to the eventual highly 
structured one, by considering the dependence on the 
averaging time of Wl^^. In FIG. 2, we plot series of 
Wl^t as functions of the measuring time (with fixed 
transient time Tt = 0) for a single trial Each line 
represents a series of for a particular value of i. 

This figure shows that units separate into two groups: 
one with large values of Wl.^^t ^'^'^ '^^^ with small values 
[ p^ .The separation becomes more distinct as the measur- 
ing time increases, although the separation process seems 
to be nearly completed by the 3 x 10^-th step, because af- 
ter this time, we do not observe the migration of any unit 
between the two groups. Also, this figure shows that the 
fluctuations of Wl^t are larger for the large W^^t group. 
This implies that W^^t fo'^ ^ unit in this group occasion- 
ally takes small values for a certain period. By contrast, 
a unit of the small W^^t group will only very rarely take 
large values of the total weight. In this sense, the small 
Wl^t group is more stable. 
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To quantify the detailed properties of the network 
structure, we digitize the connections as follows: If w^^ 
exceeds a threshold value, namely 1/{N — 1), we assign 
a connection from unit j to unit i; otherwise no connec- 
tion is assigned. This threshold 1/{N — 1) is equal to 
the connection value in the case that a unit uniformly 
connects to all the others. Hence it is a natural criterion 
for distinguishing 'strong' connections. 

Using this method, we can represent the network by a 
graph. We composed graphs from snapshots of at the 
500, 000-th step for many initial conditions. From these 
graphs, we calculated distribution of the degree of recep- 
tion and emission. The degree of reception is the number 
of connections directed at a unit (subsequently reffered 
to as "inwardly-directed" connections) and the degree of 
emission is the number of connections eminating from 
a unit (subsequently reffered to as "outwardly-directed 
connections".) The distributions of these two quantities 
are shown in FIG. 3. 

The distribution of the degree of reception has a uni- 
modal shape, with a peak at about 8 degrees. Hence, 
with regard to the inwardly-directed connections, this 
network has a single scale. This is mainly due to the com- 
petition among inwardly-directed connections, resulting 
from their normalization. 

By contrast, the distribution of the degree of emission 
has a bimodal shape, which can be decomposed into two 
components. One component is a distribution with ex- 
ponential decay, corresponding to the small Wl^^ group 
in FIG. 2. The nother component is the unimodal distri- 
bution with a peak at about 45 degrees, corresponding 
to the large Wl^^ group in FIG. 2. 

Considering the appearance of the two components in 
the distribution of the outwardly-directed connections, 
we divide the units into two groups. One group we call 
the 'core group,' which consists of units with more than 
20 connections, and the other we call the 'peripheral 
group,' which consists of all other units. With the pa- 
rameter values used here, the number of units in the core 
group is typically 14. 

With the partition of units into these two groups, the 
connections are naturally classified into four groups. The 
group to which a given connection belongs is determined 
by the groups to which the two units it connects be- 
long. With obvious identification, we call these groups 
the 'core-to-core group', the 'core-to-peripheral group', 
the 'peripheral- to-core group', and the 'peripheral-to- 
peripheral group'. 

The number and density of connections in each group 
are listed in Table 1. The most apparent characteris- 
tics seen here are that the peripheral-to-core group has 
very few members, while the core-to-peripheral group has 
many members. Also, it is senn that the density of core- 
to-core connections is quite high and that of peripheral- 
to-peripheral connection is quite low. From these results, 
we can conclude that the units in the core group interact 
strongly with each other and that the dynamics of the 
core group strongly influence the peripheral group, but 



that the dynamics of the peripheral group have almost 
no influence on the core group. 

To this point, we have investigated our system mainly 
with regard to network structure, largely ignoring the dy- 
namics of units underlying the structure formation. How- 
ever, it is clear that there must be interdependency of the 
unit dynamics and connection dynamics for the structure 
formation discussed above to occur. We have confirmed 
this directly through numerical simulations. Using the 
somewhat unnatural restriction under which connection 
weights depend on the dynamics of the units but that the 
dynamics of the units do not depend on the connection 
weights, we found that no structure ever appears. More 
detailed analysis of this point is now underway and its 
results will be presented elsewhere. 

To summarize, as a model of temporally evolving net- 
works, we have considered a network of dynamic units, 
whose dynamics are described by logistic maps and 
which are coupled to each other with variable connection 
weights. The model exhibits dynamical self-organization 
of its network structure, reflected by the state of units' 
collective behavior. Even in the parameter region where 
there is no synchronization of the unit dynamics, some 
structural order emerges. There, units spontaneously 
separate into two groups, with one group possessing espe- 
cially many outwardly-directed connections to the other 
group. 

Because of the simplicity of the model and the univer- 
sality among globally coupled maps, we believe that the 
phenomena revealed in this study are exhibited gener- 
ally by a network whose connections change in a manner 
governed by the relationships between its dynamic ele- 
ments. One example is neural networks. Though the 
conventional understanding has been that the timescale 
of the change of synaptic weights is much slower than 
that of neuronal dynamics, more and more evidence is 
being published providing evidence that synaptic change 
occurs over a wide range of timescales, from hundreds 
of milliseconds to months or years In addition, it 

is known that, in the early stage of development, axons 
arborize excessively, and eventually are trimmed under 
the influence of neuronal activity Our model seems 
to be suited to the modehng of such a situation. The 
structure formation observed in this study may provide 
a basic description of local structure formation in the 
brain, such as columnar structure. 

As mentioned at the beginning of this Letter, there 
is yet little known about dynamic networks. The sig- 
nificance of our study will be revealed as empirical data 
about dynamic networks are obtained. 
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FIG. 1. Gray scale plots of network quantities with re- 
spect to the parameters a and c, with a discretization of 0.01 
for both, (a) The activity A of the network. Brighter color 
corresponds to higher network activity. (See the text for the 
definition of the quantity A.) (b) The variance of the to- 
tal weight of outwardly-directed connections W^ut- Brighter 
color corresponds to larger variance. 
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Averaging time 

FIG. 2. The total weights of outwardly-directed connec- 
tions Wout as functions of the measuring time for the average 
connection matrix W^^ . Each curve corresponds to a single 
unit and represents a series of W^ut with different measuring 
times. The series of Wout for all units are superimposed. 
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FIG. 3. Distributions of the degrees of reception and emis- 
sion. A logarithmic scale is used for the vertical axis. The 
solid curve is the distribution of the degree of emission, and 
the broken line is that of the degree of reception. 



Group 


No. of connections 


density 


CC 


159 


0.6625 


CP 


792 


0.5893 


PC 


2 


0.0015 


pp 


234 


0.0336 



TABLE I. Number and density of connections in 
each group. Here, CC, CP, PC and PP denote the 
core-to-core, core-to-peripheral, perhipheral-to-core and pe- 
ripheral-to-peripheral groups. (See the text for the definitions 
of the groups.) 
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